{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/milvus-io/bootcamp/blob/master/bootcamp/tutorials/quickstart/hybrid_search_with_milvus.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>   <a href=\"https://github.com/milvus-io/bootcamp/blob/master/bootcamp/tutorials/quickstart/hybrid_search_with_milvus.ipynb\" target=\"_blank\">\n",
    "    <img src=\"https://img.shields.io/badge/View%20on%20GitHub-555555?style=flat&logo=github&logoColor=white\" alt=\"GitHub Repository\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hybrid Search with Dense and Sparse Vectors in Milvus\n",
    "\n",
    "<img src=\"https://raw.githubusercontent.com/milvus-io/bootcamp/master/bootcamp/tutorials/quickstart/apps/hybrid_demo_with_milvus/pics/demo.png\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this tutorial, we will demonstrate how to conduct hybrid search with [Milvus](https://milvus.io/docs/multi-vector-search.md) and [BGE-M3 model](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/BGE_M3). BGE-M3 model can convert text into dense and sparse vectors. Milvus supports storing both types of vectors in one collection, allowing for hybrid search that enhances the result relevance.\n",
    "\n",
    "Milvus supports Dense, Sparse, and Hybrid retrieval methods:\n",
    "\n",
    "- Dense Retrieval: Utilizes semantic context to understand the meaning behind queries.\n",
    "- Sparse Retrieval: Emphasizes keyword matching to find results based on specific terms, equivalent to full-text search.\n",
    "- Hybrid Retrieval: Combines both Dense and Sparse approaches, capturing the full context and specific keywords for comprehensive search results.\n",
    "\n",
    "By integrating these methods, the Milvus Hybrid Search balances semantic and lexical similarities, improving the overall relevance of search outcomes. This notebook will walk through the process of setting up and using these retrieval strategies, highlighting their effectiveness in various search scenarios."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dependencies and Environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install --upgrade pymilvus \"pymilvus[model]\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To demonstrate search, we need a corpus of documents. Let's use the Quora Duplicate Questions dataset and place it in the local directory. \n",
    "\n",
    "Source of the dataset: [First Quora Dataset Release: Question Pairs](https://www.quora.com/q/quoradata/First-Quora-Dataset-Release-Question-Pairs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell to download the dataset\n",
    "!wget http://qim.fs.quoracdn.net/quora_duplicate_questions.tsv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load and Prepare Data\n",
    "\n",
    "We will load the dataset and prepare a small corpus for search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "What is the strongest Kevlar cord?\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "file_path = \"quora_duplicate_questions.tsv\"\n",
    "df = pd.read_csv(file_path, sep=\"\\t\")\n",
    "questions = set()\n",
    "for _, row in df.iterrows():\n",
    "    obj = row.to_dict()\n",
    "    questions.add(obj[\"question1\"][:512])\n",
    "    questions.add(obj[\"question2\"][:512])\n",
    "    if len(questions) > 500:  # Skip this if you want to use the full dataset\n",
    "        break\n",
    "\n",
    "docs = list(questions)\n",
    "\n",
    "# example question\n",
    "print(docs[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use BGE-M3 Model for Embeddings\n",
    "\n",
    "The BGE-M3 model can embed texts as dense and sparse vectors. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Fetching 30 files: 100%|██████████| 30/30 [00:00<00:00, 302473.85it/s]\n",
      "Inference Embeddings: 100%|██████████| 32/32 [01:59<00:00,  3.74s/it]\n"
     ]
    }
   ],
   "source": [
    "from milvus_model.hybrid import BGEM3EmbeddingFunction\n",
    "\n",
    "ef = BGEM3EmbeddingFunction(use_fp16=False, device=\"cpu\")\n",
    "dense_dim = ef.dim[\"dense\"]\n",
    "\n",
    "# Generate embeddings using BGE-M3 model\n",
    "docs_embeddings = ef(docs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup Milvus Collection and Index\n",
    "\n",
    "We will set up the Milvus collection and create indices for the vector fields.\n",
    "\n",
    "> - Setting the uri as a local file, e.g. \"./milvus.db\", is the most convenient method, as it automatically utilizes [Milvus Lite](https://milvus.io/docs/milvus_lite.md) to store all data in this file.\n",
    "> - If you have large scale of data, say more than a million vectors, you can set up a more performant Milvus server on [Docker or Kubernetes](https://milvus.io/docs/quickstart.md). In this setup, please use the server uri, e.g.http://localhost:19530, as your uri.\n",
    "> - If you want to use [Zilliz Cloud](https://zilliz.com/cloud), the fully managed cloud service for Milvus, adjust the uri and token, which correspond to the [Public Endpoint and API key](https://docs.zilliz.com/docs/on-zilliz-cloud-console#cluster-details) in Zilliz Cloud."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pymilvus import (\n",
    "    connections,\n",
    "    utility,\n",
    "    FieldSchema,\n",
    "    CollectionSchema,\n",
    "    DataType,\n",
    "    Collection,\n",
    ")\n",
    "\n",
    "# Connect to Milvus given URI\n",
    "connections.connect(uri=\"./milvus.db\")\n",
    "\n",
    "# Specify the data schema for the new Collection\n",
    "fields = [\n",
    "    # Use auto generated id as primary key\n",
    "    FieldSchema(\n",
    "        name=\"pk\", dtype=DataType.VARCHAR, is_primary=True, auto_id=True, max_length=100\n",
    "    ),\n",
    "    # Store the original text to retrieve based on semantically distance\n",
    "    FieldSchema(name=\"text\", dtype=DataType.VARCHAR, max_length=512),\n",
    "    # Milvus now supports both sparse and dense vectors,\n",
    "    # we can store each in a separate field to conduct hybrid search on both vectors\n",
    "    FieldSchema(name=\"sparse_vector\", dtype=DataType.SPARSE_FLOAT_VECTOR),\n",
    "    FieldSchema(name=\"dense_vector\", dtype=DataType.FLOAT_VECTOR, dim=dense_dim),\n",
    "]\n",
    "schema = CollectionSchema(fields)\n",
    "\n",
    "# Create collection (drop the old one if exists)\n",
    "col_name = \"hybrid_demo\"\n",
    "if utility.has_collection(col_name):\n",
    "    Collection(col_name).drop()\n",
    "col = Collection(col_name, schema, consistency_level=\"Strong\")\n",
    "\n",
    "# To make vector search efficient, we need to create indices for the vector fields\n",
    "sparse_index = {\"index_type\": \"SPARSE_INVERTED_INDEX\", \"metric_type\": \"IP\"}\n",
    "col.create_index(\"sparse_vector\", sparse_index)\n",
    "dense_index = {\"index_type\": \"AUTOINDEX\", \"metric_type\": \"IP\"}\n",
    "col.create_index(\"dense_vector\", dense_index)\n",
    "col.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Insert Data into Milvus Collection\n",
    "\n",
    "Insert documents and their embeddings into the collection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of entities inserted: 502\n"
     ]
    }
   ],
   "source": [
    "# For efficiency, we insert 50 records in each small batch\n",
    "for i in range(0, len(docs), 50):\n",
    "    batched_entities = [\n",
    "        docs[i : i + 50],\n",
    "        docs_embeddings[\"sparse\"][i : i + 50],\n",
    "        docs_embeddings[\"dense\"][i : i + 50],\n",
    "    ]\n",
    "    col.insert(batched_entities)\n",
    "print(\"Number of entities inserted:\", col.num_entities)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Enter Your Search Query"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "How to start learning programming?\n"
     ]
    }
   ],
   "source": [
    "# Enter your search query\n",
    "query = input(\"Enter your search query: \")\n",
    "print(query)\n",
    "\n",
    "# Generate embeddings for the query\n",
    "query_embeddings = ef([query])\n",
    "# print(query_embeddings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run the Search\n",
    "\n",
    "We will first prepare some helpful functions to run the search:\n",
    "\n",
    "- `dense_search`: only search across dense vector field\n",
    "- `sparse_search`: only search across sparse vector field\n",
    "- `hybrid_search`: search across both dense and vector fields with a weighted reranker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pymilvus import (\n",
    "    AnnSearchRequest,\n",
    "    WeightedRanker,\n",
    ")\n",
    "\n",
    "\n",
    "def dense_search(col, query_dense_embedding, limit=10):\n",
    "    search_params = {\"metric_type\": \"IP\", \"params\": {}}\n",
    "    res = col.search(\n",
    "        [query_dense_embedding],\n",
    "        anns_field=\"dense_vector\",\n",
    "        limit=limit,\n",
    "        output_fields=[\"text\"],\n",
    "        param=search_params,\n",
    "    )[0]\n",
    "    return [hit.get(\"text\") for hit in res]\n",
    "\n",
    "\n",
    "def sparse_search(col, query_sparse_embedding, limit=10):\n",
    "    search_params = {\n",
    "        \"metric_type\": \"IP\",\n",
    "        \"params\": {},\n",
    "    }\n",
    "    res = col.search(\n",
    "        [query_sparse_embedding],\n",
    "        anns_field=\"sparse_vector\",\n",
    "        limit=limit,\n",
    "        output_fields=[\"text\"],\n",
    "        param=search_params,\n",
    "    )[0]\n",
    "    return [hit.get(\"text\") for hit in res]\n",
    "\n",
    "\n",
    "def hybrid_search(\n",
    "    col,\n",
    "    query_dense_embedding,\n",
    "    query_sparse_embedding,\n",
    "    sparse_weight=1.0,\n",
    "    dense_weight=1.0,\n",
    "    limit=10,\n",
    "):\n",
    "    dense_search_params = {\"metric_type\": \"IP\", \"params\": {}}\n",
    "    dense_req = AnnSearchRequest(\n",
    "        [query_dense_embedding], \"dense_vector\", dense_search_params, limit=limit\n",
    "    )\n",
    "    sparse_search_params = {\"metric_type\": \"IP\", \"params\": {}}\n",
    "    sparse_req = AnnSearchRequest(\n",
    "        [query_sparse_embedding], \"sparse_vector\", sparse_search_params, limit=limit\n",
    "    )\n",
    "    rerank = WeightedRanker(sparse_weight, dense_weight)\n",
    "    res = col.hybrid_search(\n",
    "        [sparse_req, dense_req], rerank=rerank, limit=limit, output_fields=[\"text\"]\n",
    "    )[0]\n",
    "    return [hit.get(\"text\") for hit in res]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's run three different searches with defined functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "dense_results = dense_search(col, query_embeddings[\"dense\"][0])\n",
    "sparse_results = sparse_search(col, query_embeddings[\"sparse\"][[0]])\n",
    "hybrid_results = hybrid_search(\n",
    "    col,\n",
    "    query_embeddings[\"dense\"][0],\n",
    "    query_embeddings[\"sparse\"][[0]],\n",
    "    sparse_weight=0.7,\n",
    "    dense_weight=1.0,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Display Search Results\n",
    "\n",
    "To display the results for Dense, Sparse, and Hybrid searches, we need some utilities to format the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def doc_text_formatting(ef, query, docs):\n",
    "    tokenizer = ef.model.tokenizer\n",
    "    query_tokens_ids = tokenizer.encode(query, return_offsets_mapping=True)\n",
    "    query_tokens = tokenizer.convert_ids_to_tokens(query_tokens_ids)\n",
    "    formatted_texts = []\n",
    "\n",
    "    for doc in docs:\n",
    "        ldx = 0\n",
    "        landmarks = []\n",
    "        encoding = tokenizer.encode_plus(doc, return_offsets_mapping=True)\n",
    "        tokens = tokenizer.convert_ids_to_tokens(encoding[\"input_ids\"])[1:-1]\n",
    "        offsets = encoding[\"offset_mapping\"][1:-1]\n",
    "        for token, (start, end) in zip(tokens, offsets):\n",
    "            if token in query_tokens:\n",
    "                if len(landmarks) != 0 and start == landmarks[-1]:\n",
    "                    landmarks[-1] = end\n",
    "                else:\n",
    "                    landmarks.append(start)\n",
    "                    landmarks.append(end)\n",
    "        close = False\n",
    "        formatted_text = \"\"\n",
    "        for i, c in enumerate(doc):\n",
    "            if ldx == len(landmarks):\n",
    "                pass\n",
    "            elif i == landmarks[ldx]:\n",
    "                if close:\n",
    "                    formatted_text += \"</span>\"\n",
    "                else:\n",
    "                    formatted_text += \"<span style='color:red'>\"\n",
    "                close = not close\n",
    "                ldx = ldx + 1\n",
    "            formatted_text += c\n",
    "        if close is True:\n",
    "            formatted_text += \"</span>\"\n",
    "        formatted_texts.append(formatted_text)\n",
    "    return formatted_texts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we can display search results in text with highlights:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "**Dense Search Results:**"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What's the best way to start learning robotics?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "How do I learn a computer language like java?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "How can I get started to learn information security?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What is Java programming? How To Learn Java Programming Language ?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "How can I learn computer security?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What is the best way to start robotics? Which is the best development board that I can start working on it?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "How can I learn to speak English fluently?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What are the best ways to learn French?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "How can you make physics easy to learn?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "How do we prepare for UPSC?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "\n",
       "**Sparse Search Results:**"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What is Java<span style='color:red'> programming? How</span> To Learn Java Programming Language ?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What's the best way<span style='color:red'> to start learning</span> robotics<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What is the alternative<span style='color:red'> to</span> machine<span style='color:red'> learning?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style='color:red'>How</span> do I create a new Terminal and new shell in Linux using C<span style='color:red'> programming?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style='color:red'>How</span> do I create a new shell in a new terminal using C<span style='color:red'> programming</span> (Linux terminal)<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Which business is better<span style='color:red'> to start</span> in Hyderabad<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Which business is good<span style='color:red'> start</span> up in Hyderabad<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What is the best way<span style='color:red'> to start</span> robotics<span style='color:red'>?</span> Which is the best development board that I can<span style='color:red'> start</span> working on it<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What math does a complete newbie need<span style='color:red'> to</span> understand algorithms for computer<span style='color:red'> programming?</span> What books on algorithms are suitable for a complete beginner<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style='color:red'>How</span> do you make life suit you and stop life from abusi<span style='color:red'>ng</span> you mentally and emotionally<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "\n",
       "**Hybrid Search Results:**"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What is the best way<span style='color:red'> to start</span> robotics<span style='color:red'>?</span> Which is the best development board that I can<span style='color:red'> start</span> working on it<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What is Java<span style='color:red'> programming? How</span> To Learn Java Programming Language ?"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What's the best way<span style='color:red'> to start learning</span> robotics<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style='color:red'>How</span> do we prepare for UPSC<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style='color:red'>How</span> can you make physics easy<span style='color:red'> to</span> learn<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What are the best ways<span style='color:red'> to</span> learn French<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style='color:red'>How</span> can I learn<span style='color:red'> to</span> speak English fluently<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style='color:red'>How</span> can I learn computer security<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style='color:red'>How</span> can I get started<span style='color:red'> to</span> learn information security<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style='color:red'>How</span> do I learn a computer language like java<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What is the alternative<span style='color:red'> to</span> machine<span style='color:red'> learning?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style='color:red'>How</span> do I create a new Terminal and new shell in Linux using C<span style='color:red'> programming?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style='color:red'>How</span> do I create a new shell in a new terminal using C<span style='color:red'> programming</span> (Linux terminal)<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Which business is better<span style='color:red'> to start</span> in Hyderabad<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Which business is good<span style='color:red'> start</span> up in Hyderabad<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What math does a complete newbie need<span style='color:red'> to</span> understand algorithms for computer<span style='color:red'> programming?</span> What books on algorithms are suitable for a complete beginner<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style='color:red'>How</span> do you make life suit you and stop life from abusi<span style='color:red'>ng</span> you mentally and emotionally<span style='color:red'>?</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import Markdown, display\n",
    "\n",
    "# Dense search results\n",
    "display(Markdown(\"**Dense Search Results:**\"))\n",
    "formatted_results = doc_text_formatting(ef, query, dense_results)\n",
    "for result in dense_results:\n",
    "    display(Markdown(result))\n",
    "\n",
    "# Sparse search results\n",
    "display(Markdown(\"\\n**Sparse Search Results:**\"))\n",
    "formatted_results = doc_text_formatting(ef, query, sparse_results)\n",
    "for result in formatted_results:\n",
    "    display(Markdown(result))\n",
    "\n",
    "# Hybrid search results\n",
    "display(Markdown(\"\\n**Hybrid Search Results:**\"))\n",
    "formatted_results = doc_text_formatting(ef, query, hybrid_results)\n",
    "for result in formatted_results:\n",
    "    display(Markdown(result))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quick Deploy\n",
    "\n",
    "To learn about how to start an online demo with this tutorial, please refer to [the example application](https://github.com/milvus-io/bootcamp/tree/master/bootcamp/tutorials/quickstart/apps/hybrid_demo_with_milvus)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}